agora inbox for [email protected]help / color / mirror / Atom feed
[PATCH v21 7/8] Row pattern recognition patch (tests). 144+ messages / 2 participants [nested] [flat]
* [PATCH v21 7/8] Row pattern recognition patch (tests). @ 2024-08-26 04:32 Tatsuo Ishii <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Tatsuo Ishii @ 2024-08-26 04:32 UTC (permalink / raw) --- src/test/regress/expected/rpr.out | 836 +++++++++++++++++++++++++++++ src/test/regress/parallel_schedule | 2 +- src/test/regress/sql/rpr.sql | 405 ++++++++++++++ 3 files changed, 1242 insertions(+), 1 deletion(-) create mode 100644 src/test/regress/expected/rpr.out create mode 100644 src/test/regress/sql/rpr.sql diff --git a/src/test/regress/expected/rpr.out b/src/test/regress/expected/rpr.out new file mode 100644 index 0000000000..0789e09435 --- /dev/null +++ b/src/test/regress/expected/rpr.out @@ -0,0 +1,836 @@ +-- +-- Test for row pattern definition clause +-- +CREATE TEMP TABLE stock ( + company TEXT, + tdate DATE, + price INTEGER +); +INSERT INTO stock VALUES ('company1', '2023-07-01', 100); +INSERT INTO stock VALUES ('company1', '2023-07-02', 200); +INSERT INTO stock VALUES ('company1', '2023-07-03', 150); +INSERT INTO stock VALUES ('company1', '2023-07-04', 140); +INSERT INTO stock VALUES ('company1', '2023-07-05', 150); +INSERT INTO stock VALUES ('company1', '2023-07-06', 90); +INSERT INTO stock VALUES ('company1', '2023-07-07', 110); +INSERT INTO stock VALUES ('company1', '2023-07-08', 130); +INSERT INTO stock VALUES ('company1', '2023-07-09', 120); +INSERT INTO stock VALUES ('company1', '2023-07-10', 130); +INSERT INTO stock VALUES ('company2', '2023-07-01', 50); +INSERT INTO stock VALUES ('company2', '2023-07-02', 2000); +INSERT INTO stock VALUES ('company2', '2023-07-03', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-04', 1400); +INSERT INTO stock VALUES ('company2', '2023-07-05', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-06', 60); +INSERT INTO stock VALUES ('company2', '2023-07-07', 1100); +INSERT INTO stock VALUES ('company2', '2023-07-08', 1300); +INSERT INTO stock VALUES ('company2', '2023-07-09', 1200); +INSERT INTO stock VALUES ('company2', '2023-07-10', 1300); +SELECT * FROM stock; + company | tdate | price +----------+------------+------- + company1 | 07-01-2023 | 100 + company1 | 07-02-2023 | 200 + company1 | 07-03-2023 | 150 + company1 | 07-04-2023 | 140 + company1 | 07-05-2023 | 150 + company1 | 07-06-2023 | 90 + company1 | 07-07-2023 | 110 + company1 | 07-08-2023 | 130 + company1 | 07-09-2023 | 120 + company1 | 07-10-2023 | 130 + company2 | 07-01-2023 | 50 + company2 | 07-02-2023 | 2000 + company2 | 07-03-2023 | 1500 + company2 | 07-04-2023 | 1400 + company2 | 07-05-2023 | 1500 + company2 | 07-06-2023 | 60 + company2 | 07-07-2023 | 1100 + company2 | 07-08-2023 | 1300 + company2 | 07-09-2023 | 1200 + company2 | 07-10-2023 | 1300 +(20 rows) + +-- basic test using PREV +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | nth_second +----------+------------+-------+-------------+------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | | | + company1 | 07-05-2023 | 150 | | | + company1 | 07-06-2023 | 90 | 90 | 120 | 07-07-2023 + company1 | 07-07-2023 | 110 | | | + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | + company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | | | + company2 | 07-05-2023 | 1500 | | | + company2 | 07-06-2023 | 60 | 60 | 1200 | 07-07-2023 + company2 | 07-07-2023 | 1100 | | | + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | +(20 rows) + +-- basic test using PREV. UP appears twice +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+ UP+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | nth_second +----------+------------+-------+-------------+------------+------------ + company1 | 07-01-2023 | 100 | 100 | 150 | 07-02-2023 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | | | + company1 | 07-05-2023 | 150 | | | + company1 | 07-06-2023 | 90 | 90 | 130 | 07-07-2023 + company1 | 07-07-2023 | 110 | | | + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | + company2 | 07-01-2023 | 50 | 50 | 1500 | 07-02-2023 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | | | + company2 | 07-05-2023 | 1500 | | | + company2 | 07-06-2023 | 60 | 60 | 1300 | 07-07-2023 + company2 | 07-07-2023 | 1100 | | | + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | +(20 rows) + +-- basic test using PREV. Use '*' +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP* DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | nth_second +----------+------------+-------+-------------+------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | | | + company1 | 07-05-2023 | 150 | 150 | 90 | 07-06-2023 + company1 | 07-06-2023 | 90 | | | + company1 | 07-07-2023 | 110 | 110 | 120 | 07-08-2023 + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | + company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | | | + company2 | 07-05-2023 | 1500 | 1500 | 60 | 07-06-2023 + company2 | 07-06-2023 | 60 | | | + company2 | 07-07-2023 | 1100 | 1100 | 1200 | 07-08-2023 + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | +(20 rows) + +-- basic test with none greedy pattern +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A A A) + DEFINE + A AS price >= 140 AND price <= 150 +); + company | tdate | price | count +----------+------------+-------+------- + company1 | 07-01-2023 | 100 | 0 + company1 | 07-02-2023 | 200 | 0 + company1 | 07-03-2023 | 150 | 3 + company1 | 07-04-2023 | 140 | 0 + company1 | 07-05-2023 | 150 | 0 + company1 | 07-06-2023 | 90 | 0 + company1 | 07-07-2023 | 110 | 0 + company1 | 07-08-2023 | 130 | 0 + company1 | 07-09-2023 | 120 | 0 + company1 | 07-10-2023 | 130 | 0 + company2 | 07-01-2023 | 50 | 0 + company2 | 07-02-2023 | 2000 | 0 + company2 | 07-03-2023 | 1500 | 0 + company2 | 07-04-2023 | 1400 | 0 + company2 | 07-05-2023 | 1500 | 0 + company2 | 07-06-2023 | 60 | 0 + company2 | 07-07-2023 | 1100 | 0 + company2 | 07-08-2023 | 1300 | 0 + company2 | 07-09-2023 | 1200 | 0 + company2 | 07-10-2023 | 1300 | 0 +(20 rows) + +-- last_value() should remain consistent +SELECT company, tdate, price, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | last_value +----------+------------+-------+------------ + company1 | 07-01-2023 | 100 | 140 + company1 | 07-02-2023 | 200 | + company1 | 07-03-2023 | 150 | + company1 | 07-04-2023 | 140 | + company1 | 07-05-2023 | 150 | + company1 | 07-06-2023 | 90 | 120 + company1 | 07-07-2023 | 110 | + company1 | 07-08-2023 | 130 | + company1 | 07-09-2023 | 120 | + company1 | 07-10-2023 | 130 | + company2 | 07-01-2023 | 50 | 1400 + company2 | 07-02-2023 | 2000 | + company2 | 07-03-2023 | 1500 | + company2 | 07-04-2023 | 1400 | + company2 | 07-05-2023 | 1500 | + company2 | 07-06-2023 | 60 | 1200 + company2 | 07-07-2023 | 1100 | + company2 | 07-08-2023 | 1300 | + company2 | 07-09-2023 | 1200 | + company2 | 07-10-2023 | 1300 | +(20 rows) + +-- omit "START" in DEFINE but it is ok because "START AS TRUE" is +-- implicitly defined. per spec. +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | nth_second +----------+------------+-------+-------------+------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | | | + company1 | 07-05-2023 | 150 | | | + company1 | 07-06-2023 | 90 | 90 | 120 | 07-07-2023 + company1 | 07-07-2023 | 110 | | | + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | + company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | | | + company2 | 07-05-2023 | 1500 | | | + company2 | 07-06-2023 | 60 | 60 | 1200 | 07-07-2023 + company2 | 07-07-2023 | 1100 | | | + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | +(20 rows) + +-- the first row start with less than or equal to 100 +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | 90 | 120 + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 1400 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | 60 | 1200 + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- second row raises 120% +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price) * 1.2, + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 1400 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- using NEXT +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 200 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | 140 | 150 + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 110 | 130 + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 2000 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | 1400 | 1500 + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 1100 | 1300 + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 200 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | 140 | 150 + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 110 | 130 + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 2000 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | 1400 | 1500 + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 1100 | 1300 + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- match everything +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (A+) + DEFINE + A AS TRUE +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 130 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 1300 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- backtracking with reclassification of rows +-- using AFTER MATCH SKIP PAST LAST ROW +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (A+ B+) + DEFINE + A AS price > 100, + B AS price > 100 +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | | + company1 | 07-02-2023 | 200 | 07-02-2023 | 07-05-2023 + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 07-07-2023 | 07-10-2023 + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | | + company2 | 07-02-2023 | 2000 | 07-02-2023 | 07-05-2023 + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-10-2023 + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- backtracking with reclassification of rows +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (A+ B+) + DEFINE + A AS price > 100, + B AS price > 100 +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | | + company1 | 07-02-2023 | 200 | 07-02-2023 | 07-05-2023 + company1 | 07-03-2023 | 150 | 07-03-2023 | 07-05-2023 + company1 | 07-04-2023 | 140 | 07-04-2023 | 07-05-2023 + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 07-07-2023 | 07-10-2023 + company1 | 07-08-2023 | 130 | 07-08-2023 | 07-10-2023 + company1 | 07-09-2023 | 120 | 07-09-2023 | 07-10-2023 + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | | + company2 | 07-02-2023 | 2000 | 07-02-2023 | 07-05-2023 + company2 | 07-03-2023 | 1500 | 07-03-2023 | 07-05-2023 + company2 | 07-04-2023 | 1400 | 07-04-2023 | 07-05-2023 + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-10-2023 + company2 | 07-08-2023 | 1300 | 07-08-2023 | 07-10-2023 + company2 | 07-09-2023 | 1200 | 07-09-2023 | 07-10-2023 + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- ROWS BETWEEN CURRENT ROW AND offset FOLLOWING +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w, + count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND 2 FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | count +----------+------------+-------+-------------+------------+------- + company1 | 07-01-2023 | 100 | 07-01-2023 | 07-03-2023 | 3 + company1 | 07-02-2023 | 200 | | | 0 + company1 | 07-03-2023 | 150 | | | 0 + company1 | 07-04-2023 | 140 | 07-04-2023 | 07-06-2023 | 3 + company1 | 07-05-2023 | 150 | | | 0 + company1 | 07-06-2023 | 90 | | | 0 + company1 | 07-07-2023 | 110 | 07-07-2023 | 07-09-2023 | 3 + company1 | 07-08-2023 | 130 | | | 0 + company1 | 07-09-2023 | 120 | | | 0 + company1 | 07-10-2023 | 130 | | | 0 + company2 | 07-01-2023 | 50 | 07-01-2023 | 07-03-2023 | 3 + company2 | 07-02-2023 | 2000 | | | 0 + company2 | 07-03-2023 | 1500 | | | 0 + company2 | 07-04-2023 | 1400 | 07-04-2023 | 07-06-2023 | 3 + company2 | 07-05-2023 | 1500 | | | 0 + company2 | 07-06-2023 | 60 | | | 0 + company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-09-2023 | 3 + company2 | 07-08-2023 | 1300 | | | 0 + company2 | 07-09-2023 | 1200 | | | 0 + company2 | 07-10-2023 | 1300 | | | 0 +(20 rows) + +-- +-- Aggregates +-- +-- using AFTER MATCH SKIP PAST LAST ROW +SELECT company, tdate, price, + first_value(price) OVER w, + last_value(price) OVER w, + max(price) OVER w, + min(price) OVER w, + sum(price) OVER w, + avg(price) OVER w, + count(price) OVER w +FROM stock +WINDOW w AS ( +PARTITION BY company +ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING +AFTER MATCH SKIP PAST LAST ROW +INITIAL +PATTERN (START UP+ DOWN+) +DEFINE +START AS TRUE, +UP AS price > PREV(price), +DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | max | min | sum | avg | count +----------+------------+-------+-------------+------------+------+-----+------+-----------------------+------- + company1 | 07-01-2023 | 100 | 100 | 140 | 200 | 100 | 590 | 147.5000000000000000 | 4 + company1 | 07-02-2023 | 200 | | | | | | | 0 + company1 | 07-03-2023 | 150 | | | | | | | 0 + company1 | 07-04-2023 | 140 | | | | | | | 0 + company1 | 07-05-2023 | 150 | | | | | | | 0 + company1 | 07-06-2023 | 90 | 90 | 120 | 130 | 90 | 450 | 112.5000000000000000 | 4 + company1 | 07-07-2023 | 110 | | | | | | | 0 + company1 | 07-08-2023 | 130 | | | | | | | 0 + company1 | 07-09-2023 | 120 | | | | | | | 0 + company1 | 07-10-2023 | 130 | | | | | | | 0 + company2 | 07-01-2023 | 50 | 50 | 1400 | 2000 | 50 | 4950 | 1237.5000000000000000 | 4 + company2 | 07-02-2023 | 2000 | | | | | | | 0 + company2 | 07-03-2023 | 1500 | | | | | | | 0 + company2 | 07-04-2023 | 1400 | | | | | | | 0 + company2 | 07-05-2023 | 1500 | | | | | | | 0 + company2 | 07-06-2023 | 60 | 60 | 1200 | 1300 | 60 | 3660 | 915.0000000000000000 | 4 + company2 | 07-07-2023 | 1100 | | | | | | | 0 + company2 | 07-08-2023 | 1300 | | | | | | | 0 + company2 | 07-09-2023 | 1200 | | | | | | | 0 + company2 | 07-10-2023 | 1300 | | | | | | | 0 +(20 rows) + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, + first_value(price) OVER w, + last_value(price) OVER w, + max(price) OVER w, + min(price) OVER w, + sum(price) OVER w, + avg(price) OVER w, + count(price) OVER w +FROM stock +WINDOW w AS ( +PARTITION BY company +ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING +AFTER MATCH SKIP TO NEXT ROW +INITIAL +PATTERN (START UP+ DOWN+) +DEFINE +START AS TRUE, +UP AS price > PREV(price), +DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | max | min | sum | avg | count +----------+------------+-------+-------------+------------+------+------+------+-----------------------+------- + company1 | 07-01-2023 | 100 | 100 | 140 | 200 | 100 | 590 | 147.5000000000000000 | 4 + company1 | 07-02-2023 | 200 | | | | | | | 0 + company1 | 07-03-2023 | 150 | | | | | | | 0 + company1 | 07-04-2023 | 140 | 140 | 90 | 150 | 90 | 380 | 126.6666666666666667 | 3 + company1 | 07-05-2023 | 150 | | | | | | | 0 + company1 | 07-06-2023 | 90 | 90 | 120 | 130 | 90 | 450 | 112.5000000000000000 | 4 + company1 | 07-07-2023 | 110 | 110 | 120 | 130 | 110 | 360 | 120.0000000000000000 | 3 + company1 | 07-08-2023 | 130 | | | | | | | 0 + company1 | 07-09-2023 | 120 | | | | | | | 0 + company1 | 07-10-2023 | 130 | | | | | | | 0 + company2 | 07-01-2023 | 50 | 50 | 1400 | 2000 | 50 | 4950 | 1237.5000000000000000 | 4 + company2 | 07-02-2023 | 2000 | | | | | | | 0 + company2 | 07-03-2023 | 1500 | | | | | | | 0 + company2 | 07-04-2023 | 1400 | 1400 | 60 | 1500 | 60 | 2960 | 986.6666666666666667 | 3 + company2 | 07-05-2023 | 1500 | | | | | | | 0 + company2 | 07-06-2023 | 60 | 60 | 1200 | 1300 | 60 | 3660 | 915.0000000000000000 | 4 + company2 | 07-07-2023 | 1100 | 1100 | 1200 | 1300 | 1100 | 3600 | 1200.0000000000000000 | 3 + company2 | 07-08-2023 | 1300 | | | | | | | 0 + company2 | 07-09-2023 | 1200 | | | | | | | 0 + company2 | 07-10-2023 | 1300 | | | | | | | 0 +(20 rows) + +-- JOIN case +CREATE TEMP TABLE t1 (i int, v1 int); +CREATE TEMP TABLE t2 (j int, v2 int); +INSERT INTO t1 VALUES(1,10); +INSERT INTO t1 VALUES(1,11); +INSERT INTO t1 VALUES(1,12); +INSERT INTO t2 VALUES(2,10); +INSERT INTO t2 VALUES(2,11); +INSERT INTO t2 VALUES(2,12); +SELECT * FROM t1, t2 WHERE t1.v1 <= 11 AND t2.v2 <= 11; + i | v1 | j | v2 +---+----+---+---- + 1 | 10 | 2 | 10 + 1 | 10 | 2 | 11 + 1 | 11 | 2 | 10 + 1 | 11 | 2 | 11 +(4 rows) + +SELECT *, count(*) OVER w FROM t1, t2 +WINDOW w AS ( + PARTITION BY t1.i + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A) + DEFINE + A AS v1 <= 11 AND v2 <= 11 +); + i | v1 | j | v2 | count +---+----+---+----+------- + 1 | 10 | 2 | 10 | 1 + 1 | 10 | 2 | 11 | 1 + 1 | 10 | 2 | 12 | 0 + 1 | 11 | 2 | 10 | 1 + 1 | 11 | 2 | 11 | 1 + 1 | 11 | 2 | 12 | 0 + 1 | 12 | 2 | 10 | 0 + 1 | 12 | 2 | 11 | 0 + 1 | 12 | 2 | 12 | 0 +(9 rows) + +-- WITH case +WITH wstock AS ( + SELECT * FROM stock WHERE tdate < '2023-07-08' +) +SELECT tdate, price, +first_value(tdate) OVER w, +count(*) OVER w + FROM wstock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + tdate | price | first_value | count +------------+-------+-------------+------- + 07-01-2023 | 100 | 07-01-2023 | 4 + 07-02-2023 | 200 | | 0 + 07-03-2023 | 150 | | 0 + 07-04-2023 | 140 | | 0 + 07-05-2023 | 150 | | 0 + 07-06-2023 | 90 | | 0 + 07-07-2023 | 110 | | 0 + 07-01-2023 | 50 | 07-01-2023 | 4 + 07-02-2023 | 2000 | | 0 + 07-03-2023 | 1500 | | 0 + 07-04-2023 | 1400 | | 0 + 07-05-2023 | 1500 | | 0 + 07-06-2023 | 60 | | 0 + 07-07-2023 | 1100 | | 0 +(14 rows) + +-- +-- Error cases +-- +-- row pattern definition variable name must not appear more than once +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price), + UP AS price > PREV(price) +); +ERROR: row pattern definition variable name "up" appears more than once in DEFINE clause +LINE 11: UP AS price > PREV(price), + ^ +-- subqueries in DEFINE clause are not supported +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START LOWPRICE) + DEFINE + START AS TRUE, + LOWPRICE AS price < (SELECT 100) +); +ERROR: cannot use subquery in DEFINE expression +LINE 11: LOWPRICE AS price < (SELECT 100) + ^ +-- aggregates in DEFINE clause are not supported +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START LOWPRICE) + DEFINE + START AS TRUE, + LOWPRICE AS price < count(*) +); +ERROR: aggregate functions are not allowed in DEFINE +LINE 11: LOWPRICE AS price < count(*) + ^ +-- FRAME must start at current row when row patttern recognition is used +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +ERROR: FRAME must start at current row when row patttern recognition is used +-- SEEK is not supported +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + SEEK + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +ERROR: SEEK is not supported +LINE 8: SEEK + ^ +HINT: Use INITIAL. diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule index f53a526f7c..299354c9ae 100644 --- a/src/test/regress/parallel_schedule +++ b/src/test/regress/parallel_schedule @@ -98,7 +98,7 @@ test: publication subscription # Another group of parallel tests # select_views depends on create_view # ---------- -test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combocid tsearch tsdicts foreign_data window xmlmap functional_deps advisory_lock indirect_toast equivclass +test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combocid tsearch tsdicts foreign_data window xmlmap functional_deps advisory_lock indirect_toast equivclass rpr # ---------- # Another group of parallel tests (JSON related) diff --git a/src/test/regress/sql/rpr.sql b/src/test/regress/sql/rpr.sql new file mode 100644 index 0000000000..302e2b86a5 --- /dev/null +++ b/src/test/regress/sql/rpr.sql @@ -0,0 +1,405 @@ +-- +-- Test for row pattern definition clause +-- + +CREATE TEMP TABLE stock ( + company TEXT, + tdate DATE, + price INTEGER +); +INSERT INTO stock VALUES ('company1', '2023-07-01', 100); +INSERT INTO stock VALUES ('company1', '2023-07-02', 200); +INSERT INTO stock VALUES ('company1', '2023-07-03', 150); +INSERT INTO stock VALUES ('company1', '2023-07-04', 140); +INSERT INTO stock VALUES ('company1', '2023-07-05', 150); +INSERT INTO stock VALUES ('company1', '2023-07-06', 90); +INSERT INTO stock VALUES ('company1', '2023-07-07', 110); +INSERT INTO stock VALUES ('company1', '2023-07-08', 130); +INSERT INTO stock VALUES ('company1', '2023-07-09', 120); +INSERT INTO stock VALUES ('company1', '2023-07-10', 130); +INSERT INTO stock VALUES ('company2', '2023-07-01', 50); +INSERT INTO stock VALUES ('company2', '2023-07-02', 2000); +INSERT INTO stock VALUES ('company2', '2023-07-03', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-04', 1400); +INSERT INTO stock VALUES ('company2', '2023-07-05', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-06', 60); +INSERT INTO stock VALUES ('company2', '2023-07-07', 1100); +INSERT INTO stock VALUES ('company2', '2023-07-08', 1300); +INSERT INTO stock VALUES ('company2', '2023-07-09', 1200); +INSERT INTO stock VALUES ('company2', '2023-07-10', 1300); + +SELECT * FROM stock; + +-- basic test using PREV +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- basic test using PREV. UP appears twice +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+ UP+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- basic test using PREV. Use '*' +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP* DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- basic test with none greedy pattern +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A A A) + DEFINE + A AS price >= 140 AND price <= 150 +); + +-- last_value() should remain consistent +SELECT company, tdate, price, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- omit "START" in DEFINE but it is ok because "START AS TRUE" is +-- implicitly defined. per spec. +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- the first row start with less than or equal to 100 +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- second row raises 120% +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price) * 1.2, + DOWN AS price < PREV(price) +); + +-- using NEXT +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + +-- match everything + +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (A+) + DEFINE + A AS TRUE +); + +-- backtracking with reclassification of rows +-- using AFTER MATCH SKIP PAST LAST ROW +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (A+ B+) + DEFINE + A AS price > 100, + B AS price > 100 +); + +-- backtracking with reclassification of rows +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (A+ B+) + DEFINE + A AS price > 100, + B AS price > 100 +); + +-- ROWS BETWEEN CURRENT ROW AND offset FOLLOWING +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w, + count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND 2 FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- +-- Aggregates +-- + +-- using AFTER MATCH SKIP PAST LAST ROW +SELECT company, tdate, price, + first_value(price) OVER w, + last_value(price) OVER w, + max(price) OVER w, + min(price) OVER w, + sum(price) OVER w, + avg(price) OVER w, + count(price) OVER w +FROM stock +WINDOW w AS ( +PARTITION BY company +ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING +AFTER MATCH SKIP PAST LAST ROW +INITIAL +PATTERN (START UP+ DOWN+) +DEFINE +START AS TRUE, +UP AS price > PREV(price), +DOWN AS price < PREV(price) +); + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, + first_value(price) OVER w, + last_value(price) OVER w, + max(price) OVER w, + min(price) OVER w, + sum(price) OVER w, + avg(price) OVER w, + count(price) OVER w +FROM stock +WINDOW w AS ( +PARTITION BY company +ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING +AFTER MATCH SKIP TO NEXT ROW +INITIAL +PATTERN (START UP+ DOWN+) +DEFINE +START AS TRUE, +UP AS price > PREV(price), +DOWN AS price < PREV(price) +); + +-- JOIN case +CREATE TEMP TABLE t1 (i int, v1 int); +CREATE TEMP TABLE t2 (j int, v2 int); +INSERT INTO t1 VALUES(1,10); +INSERT INTO t1 VALUES(1,11); +INSERT INTO t1 VALUES(1,12); +INSERT INTO t2 VALUES(2,10); +INSERT INTO t2 VALUES(2,11); +INSERT INTO t2 VALUES(2,12); + +SELECT * FROM t1, t2 WHERE t1.v1 <= 11 AND t2.v2 <= 11; + +SELECT *, count(*) OVER w FROM t1, t2 +WINDOW w AS ( + PARTITION BY t1.i + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A) + DEFINE + A AS v1 <= 11 AND v2 <= 11 +); + +-- WITH case +WITH wstock AS ( + SELECT * FROM stock WHERE tdate < '2023-07-08' +) +SELECT tdate, price, +first_value(tdate) OVER w, +count(*) OVER w + FROM wstock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- +-- Error cases +-- + +-- row pattern definition variable name must not appear more than once +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price), + UP AS price > PREV(price) +); + +-- subqueries in DEFINE clause are not supported +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START LOWPRICE) + DEFINE + START AS TRUE, + LOWPRICE AS price < (SELECT 100) +); + +-- aggregates in DEFINE clause are not supported +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START LOWPRICE) + DEFINE + START AS TRUE, + LOWPRICE AS price < count(*) +); + +-- FRAME must start at current row when row patttern recognition is used +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- SEEK is not supported +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + SEEK + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); -- 2.25.1 ----Next_Part(Mon_Aug_26_13_39_47_2024_878)-- Content-Type: Text/X-Patch; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="v21-0008-Allow-to-print-raw-parse-tree.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
* [PATCH v6a 3/5] disable cirrus @ 2026-05-28 17:31 Andres Freund <[email protected]> 0 siblings, 0 replies; 144+ messages in thread From: Andres Freund @ 2026-05-28 17:31 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" ^ permalink raw reply [nested|flat] 144+ messages in thread
end of thread, other threads:[~2026-05-28 17:31 UTC | newest] Thread overview: 144+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2024-08-26 04:32 [PATCH v21 7/8] Row pattern recognition patch (tests). Tatsuo Ishii <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]> 2026-05-28 17:31 [PATCH v6a 3/5] disable cirrus Andres Freund <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox